Barriers to a Wide-Area Trusted Network Early Warning System for Electric Power Disturbances

It is apparent that perturbations of the North American electric power grid follow the patterns and characteristics of Self Organized Critical (SOC) systems. Published studies show SOC systems exhibit statistical properties that may result in early warning systems predicting electric power instability and loss of reliability. However, in order for such an early warning system to exist, a trusted wide-area data network must collate information from disparate subsystems and compile that information into homogenous data records for input to a modeling system. This paper explores similarities between widearea trusted computer networks and the needs of a widearea trusted network early warning system for electric power instability. An analysis of commercial equipment used in power stations uncovers disparate access methods and protocols that inhibit interoperability, and yet that problem has been mitigated in computer networking. Recent advances lend hope that wide-area data collection and modeling of electric power system perturbations will be commonplace in the not to distant future. I. Modeling Disturbance Data The electric power grid can be modeled as a complex system of dynamic load and generation balances characterized by two types of stability. Steady-state stability describes the nominal balancing of relatively minor disturbances in load/generation fluctuations caused by normal start-up and shut-down events associated with the millions of appliances and equipment attached to the grid. Constraints on operating parameters – nominally voltage levels, current magnitudes and power flows – are used to achieve steady-state equilibrium where the generation input is matched to system losses and electrical outputs. Whereas, transient stability describes the power systems ability (or lack thereof) to absorb major disturbances and return to a relatively balanced steadystate. Load shedding, generation shedding, and regional 0-7695-1435-9/02 $ Annual Hawaii International Conference on System Sciences (HICSS-35’02) © 2002 IEEE islanding are all means to dampen the wild system oscillations evident during stages of transient stability. The North American Electric Reliability Council (NERC) has kept power system disturbance data for the years 1984 to present. There have been several efforts to model the data and although there are disagreements as to the best statistical approach, most researchers agree that the data exhibit three characteristics: 1. Nominally the data show a steady-state stability (aka, slow dynamics or slow timescale), with only mild fluctuations as the system rebalances to equilibrium after normal load/generation fluctuations. 2. Major disturbances are manifest as transient stability (aka, fast dynamics or fast timescale), with wild oscillations triggered by faults due to natural phenomena and/or sudden, unexpected system perturbations. 3. The probability distribution function of power demand/loss during cascading transient stability exhibits an exponential “power law tail” (aka heavy tail distribution) that can be linearly modeled in loglog scaling. Several models have been constructed to analyze and interpret the NERC disturbance data. Among them are Self-Organized Criticality (SOC) first proposed by Carreras, et al. [1] and later supported by collaborative work with others [2,3,4]; Highly Optimized Tolerance (HOT) proposed by Carlson and Doyle [5]; and the DCFuse model recently proposed by Chen, Thorp, and Parashar [6]. All three approaches parameterize both the slow and fast dynamics of the disturbance data, and exhibit probability distribution functions with power law scaling and power tails that can be modeled as a linear function of the power distribution exponent. Further research is needed to determine relative accuracy and benefits of each model, but initial results suggest that disturbance prediction is feasible if the right global 17.00 (c) 2002 IEEE 1 Proceedings of the 35th Hawaii International Conference on System Sciences 2002 Proceedings 0-7695-1435system conditions are accurately monitored and reported in a timely fashion. In their comparison of the NERC data to SOC sandpiles, Carreras, et al. concluded that (a) excluding weather related disturbances had little effect on the modeling, and (b) correlations between blackouts can be attributed to power system global dynamics rather than correlations in events that trigger outages [2]. The idea that random events (e.g., weather) can trigger a disturbance and still have little effect on the model seems counter-intuitive, but is easily expla ined by recognizing the “near-critical” nature of the system and “near-trip” nature of system events [3]: “If the system operates close to a ‘critical’ point, some aspects of the response of the system to random perturbation may have a universal character.” This strongly suggests that understanding the nearcritical global dynamics – and monitoring the near-trip events that occur within the system – may lead to predictive systems capable of providing warning that steady-state equilibrium is in jeopardy and the system is on the verge of slipping into transient stability. Preliminary work has affirmed this suggestion and lends support to the claim that major disturbances are somewhat predictable. In their Markov model of a DC load flow network, Dobson, et al. [4] define a system where: “Lines fail probabilistically and the consequent redistribution of power flows is calculated ... {such that} cascading line outages leading to a blackout are modeled and the lines involved in a blackout are predicted.” II. Real-Time Predictive Modeling Rapid progress in the modeling of electric power disturbance events is evident in the last two years of research from the CIN/SI federal funding program. 1 This progress lends hope that real-time predictive modeling of these events may be feasible in the near future. Such realtime modeling may enable faster acting automated and manual controls during the early stages of a disturbance in order to dampen oscillations and improve transient stability. For example, an analysis of the 1996 West Coast cascading blackout suggests that an early warning system operating within a 5-6 minute window could have 1. Complex Interactive Networks/Systems Initiative, a DOD university research initiative. 0-7695-1435-9/02 $ of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 9/02 $17.00 © 2002 IEEE initiated load shedding of 300-400 MW and/or given sufficient notification for operators in California to bring auxiliary generation on line [7,8,9]. Data from [9] shows several instances where a regional early warning system for electric power disturbances could have alerted operators to necessary actions prior to islanding, and thus preserved service for the majority of customers. These actions could have prevented the cascading outages and subsequent islanding that affected 7.5M customers in 11 U.S. states and two Canadian provinces, and cost an estimated $1.5B dollars in damages and lost service revenues. At the macro-economic level there are two approaches to implementing a very wide-area (e.g., regional or national) control/warning network: (1) strongly centralized and (2) highly decentralized. Extreme examples at both ends of the spectrum would be the centralized DOD war-room model and the decentralized Internet E-commerce model, respectively. For critical applications, such as controlling inter-regional electric power fluctuations, both approaches require very fast calculations, high speed communications, uniform or homogenized information structures, a robust communications infrastructure that is resilient to attack and natural phenomena, guaranteed quality of service levels and service agreements, and an integrated trust hierarchy (or framework). In a trusted computing network, trusted paths or channels are opened between the sender and receiver so that information can be shared without compromising the safe, reliable operation of the control functions of the computing systems at either end of the communications path. Hence, a trusted computing system requires: • Access by trusted parties • Denial of access to unauthorized persons • Sender and receiver authentication • Settings control and protection • Communications integrity and confidentiality • Accountability via access and alterations audit logging • Service policies and safeguards against denial of service • Mechanisms for both sender and receiver nonrepudiation All of these mechanisms would have to be integrated into any type of wide-area trusted network early warning system for electric power disturbances. In the next 17.00 (c) 2002 IEEE 2 Proceedings of the 35th Hawaii International Conference on System Sciences 2002 Proceedings o 0-7695-1435-9 section we list and discuss the barriers and obstacles to implementing such a system. III. Barriers to an Electric Power Disturbance Early Warning System Despite 18 years of research and development in the area of trusted networked computer systems, robust, reliable computer networks for critical applications and infrastructures are still in their adolescence. Evidence of the defensive weakness and fragility of E-Commerce, telecommunications, and finance abound in the plethora of cyber-attacks, intrusions, theft, and financial fraud conducted electronically every day throughout the United States [10]. Further, the security and reliability of military defense systems is becoming increasingly suspect due to increasing incidents of cyber-espionage and information warfare [10]. It is clear that the vulnerability of any networked computing system increases with the number of network access points enabled within that system. Thus, a wide-area early warning network for electric power disturbances suffers from the same barriers and obstacles seen in creating wide-area trusted computing networks: • Absence of a wide-area protection-level communications infrastructure • Fragility of the Internet and other telecommunications infrastructures • Lack of network quality of service guarantees and industrial strength service agreements • Immaturity, fragility, and lack of interoperability in trust frameworks • Lack of requirements and standards for reporting below-threshold disturbance anomalies • The variety of control station and substation communications protocols and their lack of interoperability • Socio-economic and political resistance to regulatory controls Optimistically, we would assume that the same technologies for mitigating risk and imple menting interoperability in computer networks could be used for control and protection in electric power systems. Unfortunately, this is only partially true. The reliability demands and time-critical nature of electric power systems place additional burdens on quality of service guarantees and high-speed authentication and trusted communications. We now elaborate on each of the above barriers. 0-7695-1435-9/02 $ f the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) /02 $17.00 © 2002 IEEE A. Absence of a Protection-Level Communications Infrastructure From their analysis of the 1996 West Coast outages, Grudinin and Roytelman [7] conclude that a nationwide, centralized control system modeled after the Russian Centralized Emergency Preventive Automatic Control (CEPAC) network would have been useful in diagnosing the power disturbances and may possibly have reduced the magnitude of the outage. In parallel arguments, (a) Birman [11] states that in order to run mission-critical applications across wide spatial areas we need to develop a Virtual Overlay Network (VON) separate from any next generation Internet network that may evolve, (b) Stahlkopf and Wilhelm [8] argue for a Wide Area Measurement System (WAMS), and (c) EPRI has proposed the Inter-control Center Communications Protocol (ICCP) as the base of an inter-regional communications infrastructure. The literal intent behind VON, WAMS, and ICCP is to segregate infrastructurerelated critical data communications (e.g., power system protection) from non-critical communications like Ecommerce. B. Fragility of Internet and Other Telecommunications Infr astructures As an alternative to a separate protection-level communications structure, several utilities and engineering services have experimented with using the Internet for access to control station data and substation equipment. While Internet access is sufficient for casual observation and maintenance planning, it is unsuitable for real-time protection. The Internet is characterized by “best-effort” non-deterministic delivery via unsecure dynamic routing, and is vulnerable to snooping, hacking, and deliberate overloading (e.g., denial of service flood attacks). These frailties preclude its use for any aspect of time-critical control applications. Other telecommunications infrastructures include the Public Switched Telephone Networks (PSTN) and leased lines forming Asynchronous Transfer Mode (ATM) networks, Frame -Relay Permanent Virtual Circuits (PVCs), and Frame -Relay Switched Virtual Circuits (SVCs). The ATM and PVC solutions have reliability and quality of service suitable for critical applications and are discussed in the next subsection. PSTN and SVC solutions have reliability and quality of service concerns, respectively, that create questions about their use in real-time applications. C. Network Quality of Service and Industrial Strength Service Agreements There are two mechanisms for ensuring quality of service guarantees over a network: (1) leased resources sufficient to handle the maximum load, and (2) packet 17.00 (c) 2002 IEEE 3 Proceedings of the 35th Hawaii International Conference on System Sciences 2002 Proceedings o 0-7695-1435prioritization that ensures priority packets are delivered at near minimum connection times. Implementing packet prioritization on proprietary networks has been done for many years, but on Ethernet networks this is still a research topic. As alternatives, several companies and organizations have implemented Ethernet TCP packets over leased ATM or PVC lines. Fortunately, these two communications mechanisms do provide quality of service guarantees suitable for time -critical applications. Unfortunately, the end-to-end TCP flow-control necessary for quality of service implementation can interfere with ATM and Frame-Relay packet construction, thereby causing an indeterminate degradation in service quality [12]. Further work is needed to better define quality of service mechanisms within ATM and Frame-Relay packets. D. Immature, Fragile Trust Frameworks At the heart of a wide-area early warning system will be the means for communicating anomalies and disturbances across spatial, economic, and governing boundaries. Hence, trusted communication between sender and receiver is a vital prerequisite before instigating any control or protective action. There are three structures, or frameworks, for establishing trusted interconnections between computing systems: (1) Internet Protocol Security (IPSec), (2) Public Key Infrastructure (PKI), and (3) an informal “web of trust.” IPSec is an effort of the Internet Engineering Task Force (IETF) to add security mechanisms to the TCP/IP layers within the Ethernet protocol. PKI is an attempt to create a world-wide infrastructure for secure communications based on asymmetric public-key cryptography. As an alternative to PKI, the Pretty Good Privacy (PGP) group has implemented and advocates an informal “web of trust” where trusted users vouch for and include others in formalized lists of who to trust. Other mechanism for establishing trust levels and trust frameworks are being explored, but by-and-large all of these efforts are focused on E-commerce and are not now sufficiently robust for electric power control systems. E. Reporting Below-Threshold Anomalies In electric power systems FERC requires utilities to report outages affecting 50,000 customers for 3 hours or more, and in telephone communications the FCC requires utilities to report failures affecting 30,000 customers for over 30 minutes. There are no reporting regulations or requirements for wireless communications and mobile network failures. In none of these three domains are there any reporting requirements for “near-critical” conditions and “near-trip” events. Such data would clearly be useful in both modeling and predicting power system disturbances, and modern digital protective relays have the capability to record and report these conditions. 0-7695-1435-9/02 $ f the 35th Annual Hawaii International Conference on System Sciences (HICSS-35’02) 9/02 $17.00 © 2002 IEEE Faults on critical supply lines, switching errors, and large apparatus failures are the most commo n causes of unstable power swings which leading to transient stability oscillations. These swings are characterized by the loss of synchronism between portions of the interconnected power systems such as that shown in Figure 2. During these swings, system voltages and currents can be quite large. The task of detecting these swings has been traditionally performed by autonomous out-of-step block and trip-measuring elements in protective relaying located in critical substations. These elements and the associated logic detect and distinguish between recoverable and nonrecoverable power swing conditions.